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This substudy in the evaluation design of the Math and Science Partnership (MSP) 
Program Evaluation examines student proficiency in mathematics and science 
for the MSPs ’schools in terms of changes across three years (2003/04, 2004/05, 
and 2005/06) and relationships with MSP-related variables using Management 
Information System data with the Annual K-12 District Survey. First, changes in 
percentages of students at or above proficient on state assessments in math and 
science were investigated by gender, ethnicity, special education, and students 
with limited English proficiency across the targeted three-year period (2003/04 
- 2005/06). The classification of MSP schools with and without focus on math or 
science during this time period was also taken into account. The results indicated 
that the MSP-related schools demonstrate sustained increase in percent of 
students at or above proficient in both math and science at the elementary and 
middle school levels, but not quite so at the high school level. Second, schools 
were examined by frequency and effect size of increase, decrease, or no change 
in student math and science proficiency. The schools with positive changes were 
in much higher numbers and higher mean effect size of change compared to 
schools with negative (or no) changes in student math and science proficiency. 
Third, the relationship between the schools ’ targeted teacher participation in 
MSP-related activities over the entire period of three years and the student math 
and science proficiency at the "end” year of this period (2005-06) was also 
investigated. This relationship was positive, yet small, at all school levels for 
mathematics, and also positive, yet much better pronounced, at the high school 
level for science. Forth, longitudinal growth trajectories in math and science 
proficiency across the three years were also investigated. The results showed 
that the schools with MSP focus on math (or science) increase at higher rate in 
math (or science) proficiency compared to those without MSP focus on math (or 
science) at the middle school level. 
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Note from the Editor. All tables and figures are presented at the end of the article. 

This study analyzes data from the MSP-Management Information System 
(MSP-MIS) initiated by NSF as a web-based data collection system. Specifically, 
the study examines student proficiency in mathematics and science for the MSPs’ 
schools in terms of changes across three years (2003/04, 2004/05, and 2005/06) and 
relationships with MSP-related variables. The purpose of the MSP-MIS is, in part, to 
assess the overall implementation of the MSP Program and to monitor the progress 
of individual MSP grants. Such implementation and monitoring are complex affairs 
because of the complexity of the MSP grants. The MSP-MIS data are self-reported 
at the school level. Each grant is a partnership, minimally involving a K-12 district 
and an institution of higher education (1HE). More often, however, multiple districts 
and multiple IHEs are engaged in a single MSP grant. The MSP-MIS collects annual 
data from all grantees, based on multiple instruments. The present study used data 
from one of the instruments, the Annual K-12 District (school-level) Survey for years 
2002/03, 2003/04, 2004/05, and 2005/06. Descriptive analyses from this survey are 
reported elsewhere (Silverstein, Bell, Frechtling, & Miyaoka, 2005). (Another MSP- 
MIS instrument provided information on an MSP’s math or science focus at the school 
level.) 

The initial year, 2002/2003, is not included in this analysis for two reasons. First, the 
number of schools that provided MIS data for 2002/03 is disproportionately smaller 
than those in the subsequent three years. For example, the number of schools with MIS 
data on math performance across all four years, 2002/03-2005/06, versus the number 
of schools with such data across the last three years, 2003/04-2005/06, is 24 versus 
214, for elementary schools, 15 versus 180, for middle schools, and 5 versus 177, for 
high schools. Second, the initial trends across the first three years, 2002/03-2004/05, 
were previously reported by MSP-PE (e.g., Dimitrov, 2006, 2007, 2008; National 
Science Foundation, 2006, 2007a). 

The following four major research questions (RQs) are addressed: 

RQ1: What are the trends in mathematics and science proficiency changes across 
the targeted three-year period (2003/04 - 2005/06) for MSP-related schools based on 
a) MIS data for all schools that reported student achievement data for any of the three 
years, and b) longitudinal MIS data — only schools with student achievement data over 
the three-year period (2003/04 -2005/06). Of particular interest is the examination of 
such trends for schools with MSP focus on the subject of interest (math or science) and 
schools without MSP focus on the subject (math or science). 

RQ2: What is the distribution of MSP-related schools across categories of change 
(increase, decrease, or no change) in math and science proficiency and what is the 
mean effect size for the categories of significant change (increase or decrease) over the 
entire three-year period of time (2003/04- 2005/06) for schools with MSP focus on the 
subject (math or science) and schools without MSP focus on the subject? 

RQ3: What are the longitudinal growth trajectories (initial school performance, 
rate of change, and interaction between them) in math and science proficiency across 
the three-year period (2003/04 - 2005/06) for schools with MSP focus on the subject 
(math or science) and schools without MSP focus on the subject? 
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RQ4: What is the relationship between schools’ targeted teacher participation in 
MSP-related activities over the three-year time period and the schools’ success in math 
and science proficiency at the end year of this time period (2005/06)? 

These four research questions address different aspects of changes in math or 
science proficiency for schools with (or without) MSP focus on math or science 
across three years (2003/04-2005/06). Specifically, a) RQ1 focuses primarily on 
the statistical significance of changes and their effect size, b) RQ2 deals with the 
distribution of schools by direction of change (decrease, no change, increase), c) RQ3 
investigates the trajectories of change in terms of initial status in math or science (i.e., 
proficiency in year 2003/04), rate of change, and possible interaction between these 
two basic parameters of growth across three years (2003/04-2005/06), and d) RQ4 
investigates the relationship between school’s targeted teacher participation in MSP- 
related activities over the three-year time period and the school’s success in math and 
science proficiency at the end year of this time period (2005/06). That is, whether a 
“critical mass” of three year targeted teacher participation in MSP-related activities 
can explain the school performance in math and science (percent of students at or 
above proficient) at the end year (2005/06). The first research question (RQ1) was 
addressed using MSP-MIS student achievement data from MSP-related schools in 
two scenarios. Namely, using schools that have reported such data for any of the three 
years 2003/04, 2004/05, and 2005/06 (in Table 2), and then using only schools that 
have reported data across these three years (see Table 3). 

Tables 2 and 3 also show that there is a substantial overlap in the number of schools 
assessed in math or science in these two scenarios. For example, the number of 
common schools in the two scenarios in mathematics at the elementary school level 
is 245 (out of 320 in 2003/04, 586 in 2004/05, and 762 in 2005/06). For science, also 
at the elementary school level, there are 1 14 common schools (out of 135 in 2003/04, 
204 in 2004/05, and 308 in 2005/06). Nevertheless, the first scenario data (Table 2) 
are used only for descriptive purposes, whereas the second scenario data (Table 3) 
are used for inferential analysis of changes in school math and science proficiency, 
including effect sizes of such changes, across all three years (2003/04-2005/06). 

The second research question (RQ2) was addressed using the longitudinal data 
from schools with MSP-MIS data on student proficiency in math (or science) across 
all three years (see Table 3). This question was answered by examining the frequency 
distribution of MSP-related schools across categories of change (increase, decrease, 
or no change) in math and science proficiency, as well as the mean effect size for the 
categories of significant change (increase or decrease) over the entire three-year period 
of time (2003/04- 2005/06). 

The third research question (RQ3) was also addressed using the longitudinal data 
from schools with MSP-MIS data on student proficiency in math (or science) across 
all three years (see Table 3). The school scores in this longitudinal analysis were 
adjusted for the school’s sample size and score variation. This was done by weighting 
the school’s proportion of students at or above proficient in math (or science) by the 
reciprocal of the standard error of this proportion: 
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Adjusted p; = Pi/s t (1) 

where p is the school’s proportion of students at or above proficient in math (or 
science) and s. is the standard error: s p . = ^p t (l- Vi/ n d with n being the sample size 
of the z'th school — that is, the number of students assessed in math (or science) in 
school i. 

With this score adjustment, if some schools have equal initial scores,/*, the larger 
the school sample size, n , the larger the factor by which the school score (proportion 
of students at or above proficient) will increase. Along with improving the reliability 
and validity by using weighted scores (e.g., Kane & Case, 2004), the score adjustment 
in this case was necessary because the growth analysis involves the school means and, 
therefore, averaging proportions that come from schools with different sample size 
would produce misleading results. After the adjustment, the square root transformation 
was applied to the resulting scores, with the purpose to reduce the (positive) skewness 
of their original distributions, thus improving the technical conditions required with 
this type of longitudinal growth modeling (e.g., Snedecor & Cochran, 1989; Stevens, 
2002). The square root transformation makes the data distribution more suitable for 
the analytic procedures involved in the growth analysis with RQ3 and does not lead 
to problems with validity of interpretations. The relationship between the original and 
adjusted proportions was found to be positive monotonic with a Pearson correlation of 
.73 between them. It is important to emphasize in this regard that the main purpose of 
RQ3 is to examine growth trajectories in math and science proficiency for two groups 
of schools — with or without MSP focus on math (or science) — not to compare 
these two groups of schools on their original percent of student proficiency; (such 
comparisons are addressed, from different angles, with research questions RQ1 and 
RQ2). 

Finally, the fourth research question (RQ4) was addressed using schools for which 
MSP-MIS data were available on targeted teacher participation at any of the three 
years (2003/04-2005/06) and student achievement data for the last year (2005/06). 
As alluded to earlier, the idea was to investigate the relationship between the school’s 
“critical mass” of targeted teacher participation in MSP-related activities over all three 
years and student math and science proficiency at the end of this time period. It is 
important to note also that the variable “targeted teacher participation in MSP-related 
activities” is not involved in the previous three research questions. 

Tables 2 and 3 summarize the information about the data that have been used in 
statistical analyses related to each of the research questions addressed in this study. 
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Method 


Data 

From the Annual K-12 District Survey, the data used in this paper covered schools 
with available data for the four research questions as described in the previous section. 
Table 2 provides data on number of schools for which MSP-MIS data on student math 
or science proficiency were available for any of the three years (2003/04, 2004/05, 
and 2005/06), number of students in these schools who had taken the state assessment 
in math or science, n, and number of students who "pass" (at or above proficient) the 
assessment. The data are also provided by gender, ethnicity, special education students, 
and limited English proficiency students. Table 2 shows, for example, that the highest 
relative sample representation of schools is for mathematics at the elementary school 
level. Table 3 includes only schools with MSP-MIS student achievement data across 
all three years (2003/04-2005/06). 

Variables and Scales 

There are three main variables investigated in this school-level MSP-MIS study: 

1. Student achievement — the proportion of students at or above proficient on 
state assessments in mathematics and science, calculated by the number of students 
attaining proficiency divided by the total number of students taking the test; 

2. Targeted teacher participation in MSP-related activities — this variable is 
identified in MSP-MIS by the condition that 30 percent or more of a school's 
targeted teachers participated in 30 or more hours of MSP-sponsored activities 
during a single school year. Given the binary scale (1 if the condition was met, 
and 0 otherwise), the score for any school on this specific variable over three 
school years (2003/04, 2004/05, and 2005/06) may vary from zero to three (0 = the 
condition was not met during any of the three years, and 3 = the condition was met 
all three years); and 

3. MSP focus on math (or science) for each school (0 = No, 1 = Yes), with "yes" 
meaning that the MSP indicated such a focus in any of the three years being studied. 

Statistical Analysis 

All research questions were addressed by school level (elementary, middle, and 
high school). To address RQ1, longitudinal analyses were conducted to compare 
schools with an MSP focus on math (or science) versus schools without such focus 
on trends and effect size of changes in percent of students at or above proficient. 
Cohen's effect size (ES) index for a difference in two proportions, h (Cohen, 1988), 
was calculated for each school with a statistically significant change (increase or 
decrease). The h effect size for the difference in two proportions, say P | - P v is: 
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h = 2arcsin^[p[ - 2arcsin^fP^ . The magnitude of the effect size is operationally 
defined as small ( h = .20), medium ( h = .50), and large ( h = .80) effect size (Cohen, 
1988, p. 181). 

To address RQ2, each school was assigned to one of three categories of change in 
terms of percent of students at or above proficient in math or science: increase — if 
the school has a statistically significant positive change, decrease — if the school has 
a statistically significant negative change, and no change — if the school’s change 
was not statistically significant. The frequency distribution of schools by direction of 
change (increase, decrease, no change) in math and science proficiency was examined 
by schools with or without MSP focus on math (or science). The changes across three 
school years were measured by the differences in percent of students at or above 
proficient on state assessments in mathematics and science from 2003/04 to 2004/05 
( one-year immediate change) and from 2003/04 to 2005/06 ( two-year sustained 
change). 

To address RQ3, longitudinal growth modeling was applied to adjusted scores 
of school proficiency in math and science to investigate the initial status (intercept) 
and rate of change (slope), as well as possible interaction between them, in growth 
trajectories of school proficiency in math and science across all three years (2003/04- 
2005/06). The role of schools with (or without) MSP focus on the respective subject 
matter (math or science) was also taken into account with this longitudinal growth 
analysis. Longitudinal growth modeling (LGM; e.g., Muthen, 2004) was employed 
with the individual schools being the units of analysis, the square root of the adjusted 
school proportion of students at or above proficient (see Equation 1) being the outcome 
variable across three years (2003/04, 2004/05, and 2005/06), and the school variable 
"MSP focus on math or science" (0 = No, 1 = Yes) being a background variable. 
Graphically, the LGM used in this study is depicted in Figure 1. The longitudinal 
growth analysis was conducted separately for math and science at each (elementary, 
middle, and high) school level using the computer program M plus (Muthen & Muthen, 
2007). 

To address RQ4, the Pearson product-moment correlation was used to investigate 
the relationship between the school's targeted teacher participation in MSP-related 
activities over the time period of all three years and student math and science 
proficiency at the end of this time period (2005/06). This analysis was conducted 
separately for math and science at each (elementary, middle, and high) school level. 

Results 

The results are reported in four parts representing the four research questions (RQ1, 
RQ2, RQ3, and RQ4) addressed in this MSP-PE substudy. 
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Trends and Effect Sizes of Changes in Math and Science Proficiency 

This section provides results related to the first research questions, RQ1: “What 
are the trends in mathematics and science proficiency changes across the targeted 
three-year period (2003/04 - 2005/06) for MSP-related schools based on MIS data 
for all schools that reported student achievement data for any of the three years, and 
longitudinal MIS data — only schools with student achievement data across all three 
years (2003/04-2005/06). Of particular interests is the examination of such trends 
for schools with MSP focus on the subject of interest (math or science) and schools 
without MSP focus on the subject (math or science).” 

The results are presented separately for student achievement in mathematics and 
science. The change of each school in percent of students at or above proficient in 
math (or science) is tested for statistical significance using 90% confidence intervals 
(90% Cl) for the change. The choice of 90% Cl over 95% Cl was guided by the 
principle of increasing test power. 

Mathematics. Figures 2 and 3 (upper panels) show the percent of students at or 
above proficient on state assessments in mathematics by school level (elementary, 
middle, and high) for all schools with MSP-MIS student achievement data at any 
of the three years (2003/04-2005/06) and only for schools with MSP-MIS student 
achievement data across all three years, respectively. The trends of school changes 
in math proficiency delineated in these two exhibits are very similar due to the fact 
that the data used for Figure 3 (upper panel) is a substantial subset of the data used 
for Figure 2 (upper panel) (see also Tables 1 and 2). Therefore, the school data used 
for Figure 3 (upper panel), that is, student achievement data available across all three 
years, were also used for inferential comparisons and calculation of effect sizes for 
school changes in math proficiency across the three years (2003/04-2005/06) (see 
Table 3). 

Figures 2 and 3 (upper panels) also show that there is a sustained increase in math 
proficiency at the elementary and middle school levels, but not at the high school 
level — specifically, there is an initial decrease (2003/04-2004/05) after which the 
math proficiency for high schools remains stable. The results by schools with (or 
without) MSP focus on math are presented with Table 4 and Figure 4. Clearly, the 
elementary and middle schools with MSP focus on math show a consistent increase 
in math proficiency, with the largest effect size {ES = +.35) for the sustained increase 
from 2003/04 to 2005/06. Conversely, the elementary and middle schools without 
MSP focus on math show an overall decrease in math proficiency (with the exception 
of a slight initial increase, ES = +.09, for middle schools). At the high school level, 
however, the math proficiency change is not in favor of schools with MSP focus on 
math. Specifically, there is a small decrease for these schools versus a small increase 
for high schools without MSP focus on math. 

By gender, the results in Table 5 show that the largest (2003/04-2005/06) increase 
in math proficiency for both males and females is at the elementary school level, with 
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a close to medium effect size ( ES = +.35). By ethnicity, the results in Table 6 show 
that the largest (2003/04-2005/06) increase in math proficiency is for the elementary 
school with MSP focus on math — (small to medium) effect size for African- 
American students (ES = +.37), Hispanic students (ES = +.37), and students who 
have not reported their ethnicity (ES = +. 31). An exception is the sizable increase in 
math proficiency (medium to high effect size: ES = +.54) at the high school level for 
students in the "other" grouping by ethnicity from the schools without MSP focus on 
math. Further, the results in Table 7 show that a) for special education students, the 
overall positive change in effect size is in favor of schools with MSP focus, and b) this 
trend is even stronger for students with limited English proficiency at the elementary 
and middle school levels, but not at the high school level. 

Science. Figures 2 and 3 (lower panels) show that there is a substantial increase 
in science proficiency at the elementary school level, less pronounced increase for 
the middle schools, and an initial decrease, followed up by a very small increase, at 
the high school level. The results by schools with (or without) MSP focus on science 
are presented with Table 8 and Figure 5. The effect size results in Table 8 show that, 
overall, the schools with MSP focus on science do better than those without MSP focus 
on science at the elementary and middle school levels, but this is not the case at the 
high school level. High schools with MSP focus on science exhibit a close to medium 
decrease (ES = -.36), whereas high schools without MSP focus on science exhibit a 
small increase (ES = +.14) in science proficiency (2003/04-2005/06). Note that the 
comparison by “percent proficient students” can be misleading due to the much larger 
sample size of students (and schools) for schools with MSP focus on science compared 
to MSP schools without focus on science. The effect size takes this into account and 
represents a more valid scale for comparison of changes in student proficiency. 

By gender, the results in Table 9 show that the largest (2003/04-2005/06) increase 
in science proficiency is for the elementary schools with MSP focus on science, with 
small effect size for both males and females (ES = +.21). By ethnicity, the results in 
Table 10 show that the largest (2003/04-2005/06) increase in science proficiency is 
for schools with MSP focus on science . There is an increase of medium effect size 
for the African-American students (ES= +.47) and Asian students (ES = +. 42), at the 
elementary school level, and for Asian students at the middle school level (ES = +.36). 

For special education students, the largest (2003/04-2005/06) increase in science 
proficiency is for the middle schools with MSP focus on science (ES = +.56) (see 
Table 11). For students with limited English proficiency, the largest (2003/04-2005/06) 
increase in science proficiency is at the middle school level, but with ES +.56 for 
schools without MSP focus on science and ES = +.30 for schools with MSP focus on 
science. There is a similar trend at the elementary school level for these students, with 
ES = +.30 for schools without MSP focus on science and ES = +.21 for schools with 
MSP focus on science. However, there is no change in science proficiency at the high 
school level for these students (see Table 11). 
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Schools by Direction of Change in Math and Science Proficiency 

The results in this section relate to the second research question, RQ2: “What is the 
distribution of MSP-related schools across categories of change (increase, decrease, 
or no change) in math and science proficiency and what is the mean effect size for 
the categories of significant change (increase or decrease) over the entire three-year 
period of time (2003/04- 2005/06) for schools with MSP focus on the subject (math or 
science) and schools without MSP focus on the subject?” 

Specifically, this section provides information about the percentage of schools by 
direction of change (increase, decrease, no change) in math and science proficiency 
over a two-year period (2003/04-2005/06), separately for schools with and without 
MSP focus on math (or science) — see Figures 6, 7 and 8, for math, and Figures 9, 10, 
and 11, for science. 

Clearly, the percentage of schools with a two-year increase is much higher than the 
percentage of schools with a two-year decrease at all school levels for both math and 
science. For schools that fall into the "increase" category, the percentage of schools 
with focus on math (or science) is higher than the percentage of schools without focus 
on math (or science) at the elementary and middle school levels for both math and 
science (see Figures 6, 7, 9, and 10). This, is not the case at the high school level 
(Figures 8 and 11). 

Longitudinal Growth Trajectories in School Math and Science Proficiency 

The results in this section relate to the third research questions, RQ3: “What are 
the longitudinal growth trajectories (initial school performance, rate of change, and 
interaction between them) in math and science proficiency across the three-year period 
(2003/04 - 2005/06) for schools with MSP focus on the subject (math or science) and 
schools without MSP focus on the subject?” 

The longitudinal growth model (LGM) of changes in school math and science 
proficiency across three years (2003/04-2005/06) is depicted in Figure 1. The results 
are summarized in Table 12. The unit of measurement are individual schools, the 
school score is the adjusted proportion of students at or above proficient (see Equation 
1), and the school "MSP focus on math (or science)" is a background variable (0 = No, 
1 = Yes). 

The results for tests of model fit in Table 12 show that the LGM model fits the 
school data fairly well, given the following three criteria of a good model fit used in 
this study: Comparative Fit Index (CFI > .95), Tucker-Lewis Index (TLI > .95), and 
Standardized Root Mean Square Residual (SRMR < .06). For the estimates of the CFI, 
for example, with the exception of a slightly lower CFI at the elementary school level 
(.844), all CFIs vary from .959 to .999 — see Table 12. 

Given the coding (0 = No, 1 = Yes) for the school variable "MSP focus on math (or 
science)," the statistically significant coefficients in the column "Initial Status on MSP 
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Focus" in Table 12 indicate that a) the schools with MSP focus on math have higher 
initial status (higher adjusted proficiency score in 2003/04) than those without MSP 
focus on math at the elementary and high school levels (0.33 and 0.37), but not on the 
middle school level (-1.46); and b) the schools with MSP focus on science have lower 
initial status (lower adjusted proficiency score in 2003/04) than those without MSP 
focus on science at the elementary school level (-1.63). 

The statistically significant positive coefficients in the column "Rate of Change on 
MSP Focus" in Table 12 show that a) the schools with MSP focus on math increase at 
higher rate in math proficiency compared to those without MSP focus on math at the 
middle school level (0.25), and b) the schools with MSP focus on science increase at 
higher rate in science proficiency compared to those without MSP focus on science 
at the middle school level. Still in Table 12, the statistically significant negative 
correlation coefficient (-.53) in the column “Initial Status correlated with Rate of 
Change” indicates that middle schools with lower initial proficiency in math increase 
at a higher rate. On the other hand, the statistically significant positive correlation 
coefficient (.25) shows that high schools with higher initial proficiency in science 
increase with higher rate. 

Relationship Between Targeted Teacher Participation in MSP-related Activities and 
Student Proficiency in Math and Science 

The results in this section relate to the fourth research question, RQ4: “What is the 
relationship between schools’ targeted teacher participation in MSP-related activities 
over the three-year time period and the schools’ success in math and science proficiency 
at the end year of this time period (2005/06)?” 

Specifically, this section provides results about the relationship between the targeted 
teacher participation in MSP-related activities over the span of three years (2003/04- 
2005/06) and the student proficiency in math and science at the end year (2005/06). 
The Pearson product-moment correlation coefficients for this relationship at the 
elementary, middle, and high school levels are provided in Table 13. The presence (or 
lack) of statistical significance for these coefficients and their magnitudes reveals that 
the relationship between the targeted teacher participation in MSP-related activities 
and student proficiency is statistically significant and positive (yet, small) at all school 
levels for mathematics, and statistically significant and well pronounced (r = .473) at 
the high school level for science. 


Discussion 

This study examines intermediate trends in MSP-related changes in student math 
and science proficiency using MSP-MIS data with the Annual K- 1 2 District Survey for 
three years, 2003/04, 2004/05, and 2005/06. The results are summarized by the topics 
of the four research questions addressed in this study. 
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Trends of Changes in Math and Science Proficiency 

The MSP-related schools demonstrate sustained increase in percent of students at 
or above proficient in both math and science at the elementary and middle school levels 
across years 2003/04, 2004/05, and 2005/06. This, however is not the case at the high 
school level, with an initial decrease (2003/04-2004/05) after which the proficiency 
for high schools remains stable for both math and science. The elementary and middle 
schools with MSP focus on math show a consistent increase in math proficiency, 
with the largest effect size for the sustained increase from 2003/04 to 2005/06 at the 
elementary school level. Conversely, the schools without MSP focus on math show an 
overall decrease in math proficiency at the elementary and middle school levels. At 
the high school level, however, the math proficiency change is not in favor of schools 
with MSP focus on math. There is a small decrease for high schools with MSP focus 
on math versus a small increase for high schools without MSP focus on math over the 
period from 2003/04 to 2005/06. 

By gender, the largest (2003/04-2005/06) increase in both math and proficiency is 
at the elementary school level, with the same magnitude for both males and females 
— specifically, a close to medium effect size math and a small effect size in science. 
By ethnicity, the largest (2003/04-2005/06) increase in student proficiency is at the 
elementary school level — for African-American students and Hispanic students in 
math, and for African American students and Asian students in science. At the middle 
school level, the increase in math proficiency is relatively small and about the same for 
all ethnic groups. A close to medium increase in science proficiency for Asian students 
is followed by a small increase for African-American students, and negligible increase 
for White and Hispanic students. 

For special education students, the largest (2003/04-2005/06) increase in math 
proficiency, with a small effect size, is at the elementary school level, whereas the 
largest increase in science for these students is at the middle school level, with a 
medium to large effect size. For students with limited English proficiency, the largest 
(2003/04-2005/06) increase in math proficiency, with a medium effect size, is at the 
elementary school level, whereas the largest increase in science proficiency for these 
students is at the middle school level, with a small to medium effect size. 

Schools by Direction of Change in Math and Science Proficiency 

For both math and science, the percentage of schools with an increase in student 
proficiency is higher than that with a decrease in student proficiency at all school 
levels over the period from 2003/04 to 2005/06. Also, for schools that fall into the 
"increase" category, the percentage of schools with MSP focus on math (or science) is 
higher than that of schools without MSP focus on math (or science) at the elementary 
and middle school levels for math (or science). This, however, is not the case at the 
high school level. 
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Longitudinal Growth Trajectories in School Math and Science Proficiency 

The schools with MSP focus on math have higher initial (2003/04) proficiency 
in math than those without MSP focus on math at the elementary and high school 
levels, but not on the middle school level. On the other side, the schools with MSP 
focus on science have lower initial proficiency in science than those without MSP 
focus on science at the elementary school level. The schools with MSP focus on 
math (or science) increase at higher rate in math (or science) proficiency compared 
to those without MSP focus on math (or science) at the middle school level. Middle 
schools with lower “start” (initial proficiency) in math increase at a higher rate in 
math proficiency across the three years (2003/04-2005/06). High schools with higher 
“start” (initial proficiency) in science increase with higher rate in science proficiency 
across the three years (2003/04-2005/06). 

Relationship Between Targeted Teacher Participation in MSP-related Activities and 
Student Proficiency in Math and Science 

The relationship between the targeted teacher participation in MSP-related activities 
and student proficiency is positive (yet, small) at all school levels for mathematics, and 
positive, and better pronounced, at the high school level for science. 

Limitations and Upcoming Analyses 

The results in this study must be interpreted with understanding of limitations that 
stem from restricted MIS data with the Annual K-12 District Survey. One limitation, 
for example, is the lack of matching data from "control" schools (not involved in 
MSP) to evaluate the degree to which the changes in students' proficiency in math and 
science can be attributed to school participation in MSP. That is why this study does 
not engage in testing a hypothesis about the degree to which the delineated trends 
in math and science performance of MSP-related schools are different from trends 
that may exist in non-MSP related schools. A strong insight in this regard, however, 
is provided by the comparisons of MSP-related schools with and without MSP focus 
on math (or science) on different aspects of changes in math (or science) proficiency 
across the three years — percent of students at or above proficient, distribution of 
schools by direction of change (decrease, no change, increase), and growth trajectories 
(initial status in proficiency, rate of change, and interaction between them). Additional 
evidence about explanatory effects of MSP-related activities in schools on student 
proficiency in math and science is sought through the fourth research question by 
analyzing the correlation between the targeted teacher participation in MSP-related 
activities and student proficiency. Triangulations with findings in other MSP-PE 
substudies that control for MSP participation of schools (e.g., Wong & Socha, 2008) 
may provide more evidence on the role of MSP factors in the math and science 

108 


Intermediate Trends 


proficiency of MSP-related schools. 

Another potential limitation stems from the lack of MIS data that can be used to 
equate school proficiency measures in math and science across states. It should be 
noted, however, that mapping state performance standards on to a common scale 
(e.g., using NAEP data) is a difficult task still challenging the research on large-scale 
performance analyses (e.g., Braun & Qian, 2007; McLaughlin & Bandeira de Mello, 
2003). The purpose of such equating is to take into account differences (in content and 
passing standards) among state assessments in math and science for the comparison 
of states on a common scale. Such comparisons, however, are not targeted in this 
study. Instead, the focus here is on changes and growth trajectories in student math and 
science proficiency and its relationship with school's targeted teacher participation in 
MSP-related activities. 

When necessary, the aggregation of schools (e.g., by elementary, middle, and 
high school level) was done not by averaging the proportions of students at or above 
proficient across schools, but by aggregating the number of students assessed and 
the number of those who "pass" (at or above proficient) thus producing a "clean" 
measure of student proficiency at the aggregated school level. Likewise, the measure 
of school proficiency by direction of change (decrease, no change, increase) in math 
or science proficiency, used with RQ2, is based on testing for statistical significance of 
the change for each school, and not on aggregated proportions across schools. When 
averaging of proportions was necessary with the growth modeling in RQ3, it was 
done after adjusting the proportions for school size and variability in math and science 
proficiency. 

In upcoming analyses with the continuation of this study, efforts will be directed 
in reducing validity threats associated with aggregation of student achievement 
trends across states — e.g., through a) mapping the aforementioned binary scores of 
change in school math or science proficiency on (IRT derived) scale, b) weighting the 
proportions of students at or above proficient in math or science, c) using standardized 
effect sizes, and d) mapping state performance standards on to a common scale when 
appropriate data (collected outside MIS) is available. Additional analyses that can 
counteract the limitations with this study are also next steps in the MSP-PE agenda. 
Such analyses (e.g., using math and science course credit teacher training data) can 
further expand our understanding of the relationship between MSP-participation and 
student math and science achievement. 

In conclusion, despite limitations in scope and depth of the analysis in this study, 
due primarily to data restrictions with the MIS Annual K-12 District Survey, the 
results indicate promising trends and relationships between student proficiency in 
mathematics and science and MSP-related variables. 
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Table 1 

Data Sets Used in the Statistical Analysis by Research Questions 


Research Question 

Data 

RQ1: What is the distribution of MSP-related 
schools across categories of change (increase, 
decrease, or no change) in math and science 
proficiency and what is the mean effect size for 
the categories of significant change (increase 
or decrease) over the entire three-year period 
of time (2003/04- 2005/06) for schools with 
MSP focus on the subject (math or science) and 
schools without MSP focus on the subject? 

MSP-MIS student achievement 
data from MSP-related schools 
in two scenarios: a) using 
schools that have reported such 
data for any of the three years 
(Table 2), and b) using only 
schools that have reported data 
across all three years (Table 3). 

RQ2: What is the distribution of MSP-related 
schools across categories of change (increase, 
decrease, or no change) in math and science 
proficiency and what is the mean effect size for 
the categories of significant change (increase 
or decrease) over the entire three-year period 
of time (2003/04- 2005/06) for schools with 
MSP focus on the subject (math or science) and 
schools without MSP focus on the subject? 

Longitudinal data from schools 
with MSP-MIS data on student 
proficiency in math (or science) 
across all three years (Table 3). 

RQ3 : What are the longitudinal growth 
trajectories (initial school performance, rate of 
change, and interaction between them) in math 
and science proficiency across the three-year 
period (2003/04 - 2005/06) for schools with 
MSP focus on the subject (math or science) and 
schools without MSP focus on the subject? 

Longitudinal data from schools 
with MSP-MIS data on student 
proficiency in math (or science) 
across all three years (Table 
3). The school scores were 
adjusted for the school’s sample 
size and score variation. 

RQ4: What is the relationship between schools’ 
targeted teacher participation in MSP-related 
activities over the three-year time period 
and the schools’ success in math and science 
proficiency at the end year of this time period 
(2005/06)? 

Schools for which MSP-MIS 
data that were available on 
targeted teacher participation 
at any of the three years 
(2003/04-2005/06) and student 
achievement data for the 
last year of this time period 
(2005/06). 
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Table 2 

MSP-MIS Cross-Sectional Data for Number of Schools, Number of Students Assessed 
and Number of Students at or Above Proficient at State Assessments in Mathematics 
and Science Across Three School Years: 2003/04, 2004/05, and 2005/06 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

All students 

2003/04 

n =53363 
pass = 25288 
320 schools 

n =98270 
pass = 35633 
227 schools 

n =97675 
pass = 39774 
213 schools 

n = 10942 
pass = 3515 
135 schools 

n =20682 
pass = 8500 
96 schools 

n =46026 
pass = 23858 
130 schools 

2004/05 

n =97534 
pass = 59417 
586 schools 

n = 195131 
pass = 81836 
358 schools 

it = 166068 

pass = 59971 
312 schools 

n = 17826 
pass = 8208 
204 schools 

n =52907 
pass = 30870 
192 schools 

n = 104732 
pass = 38063 
210 schools 

2005/06 

n = 164369 
pass = 107039 
762 schools 

n =318916 
pass = 152851 
521 schools 

n = 199838 
pass = 72493 
381 schools 

n =33859 
pass = 20388 
308 schools 

it =93200 
pass = 47019 
275 schools 

n = 121547 
pass = 46884 
251 schools 


Males 

2003/04 

n =26975 
pass = 12602 
(320 schools) 

rt =49878 
pass = 17866 
(227 schools) 

n = 49044 
pass = 20049 
(213 schools) 

n =5348 
pass = 1686 
(135 schools) 

n = 10513 
pass = 4417 
(96 schools) 

n =23015 
pass = 12165 
(130 schools) 

2004/05 

n =44102 
pass = 26046 
(490 schools) 

n =81262 
pass = 30874 
(293 schools) 

n =78859 
pass = 27307 
(266 schools) 

n =7921 
pass = 3253 
(193 schools) 

n = 15086 
pass = 7627 
(142 schools) 

n =49283 
pass = 17696 
(173 schools) 

2005/06 

n =78846 
pass = 4961 1 
(704 schools) 

n = 143821 
pass = 69459 
(471 schools) 

n =88549 
pass = 34086 
(345 schools) 

n = 16382 
pass = 9850 
(285 schools) 

n =44461 
pass = 23316 
(255 schools) 

n =58106 
pass = 23317 
(227 schools) 

Females 

2003/04 

n =26064 
pass = 12553 
(320 schools) 

n =48361 
pass = 17749 
(227 schools) 

n =48245 
pass = 19476 
(213 schools) 

it =5350 
pass = 1720 
(135 schools) 

n = 10156 
pass = 4077 
(96 schools) 

n =22853 
pass= 11589 
(130 schools) 

2004/05 

n =42317 
pass = 25515 
(490 schools) 

n =79609 
pass = 30329 
(293 schools) 

n =77105 
pass = 265 1 5 
(266 schools) 

n =7700 
pass = 3120 
(193 schools) 

n = 14535 
pass = 7103 
(142 schools) 

n =48086 
pass = 16421 
(173 schools) 

2005/06 

n =75919 
pass = 48491 
(704 schools) 

n = 140155 
pass = 69807 
(471 schools) 

n =87706 
pass = 33688 
(345 schools) 

n = 15960 
pass = 9750 
(285 schools) 

n =43851 
pass = 22309 
(254 schools) 

it =57726 
pass = 21908 
(225 schools) 
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Table 2 (continued) 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

White 

2003/04 

n = 12333 
pass = 9318 
(320 schools) 

n =26345 
pass = 17108 
(227 schools) 

n =20916 
pass = 13044 
(213 schools) 

n =4476 
pass = 1998 
(135 schools) 

n =8798 
pass = 5560 
(96 schools) 

n = 13160 
pass = 9535 
(130 schools) 

2004/05 

n =27473 
pass = 2161 1 
(495 schools) 

n =47433 
pass = 3255 1 
(329 schools) 

n =34966 
pass = 21716 
(283 schools) 

n =5984 
pass = 3606 
(193 schools) 

n = 14890 
pass = 10435 
(170 schools) 

n =22800 
pass = 15768 
(190 schools) 

2005/06 

n =62575 
pass = 46465 
(704 schools) 

n =99768 
pass = 68278 
(467 schools) 

n =39926 
pass = 25129 
(329 schools) 

n = 10174 
pass = 7200 
(281 schools) 

n =22604 
pass = 16559 
(241 schools) 

n =23471 
pass = 16540 
(209 schools) 

African American 

2003/04 

n =6668 
pass = 2386 
(320 schools) 

n = 13227 
pass = 3032 
(227 schools) 

n =8394 
pass = 2292 
(213 schools) 

n = 1320 
pass = 229 
(135 schools) 

n =5031 
pass = 875 
(96 schools) 

n =5296 
pass = 2445 
(130 schools) 

2004/05 

n = 14653 
pass = 7037 
(4 1 8 schools) 

n =24594 
pass = 6743 
(296 schools) 

n = 16843 
pass = 3936 
(259 schools) 

n =2340 
pass = 737 
(125 schools) 

n =6463 
pass = 1 843 
(141 schools) 

n = 11658 
pass = 3390 
(166 schools) 

2005/06 

n =38796 
pass = 24190 
(6 1 6 schools) 

n =48151 
pass = 18776 
(417 schools) 

n = 18756 
pass = 5756 
(291 schools) 

n = 12669 
pass = 8774 
(205 schools) 

n =21116 
pass = 8302 
(196 schools) 

n = 13106 
pass = 4687 
(177 schools) 

Hispanic/Latino 

2003/04 

n =30588 
pass = 11514 
(320 schools) 

n =48220 
pass = 9555 
(227 schools) 

n =61155 
pass = 20766 
(213 schools) 

n =3835 
pass = 803 
(135 schools) 

n =4386 
pass = 1027 
(96 schools) 

n =22838 
pass = 9148 
(130 schools) 

2004/05 

n =44831 
pass = 24143 
(586 schools) 

n = 102259 
pass = 31277 
(358 schools) 

n = 100665 
pass = 2861 1 
(312 schools) 

n =8178 
pass = 3327 
(204 schools) 

n =26366 
pass = 16635 
(192 schools) 

n =60487 
pass = 14577 
(210 schools) 

2005/06 

n =46059 
pass = 23164 
(762 schools) 

n = 123816 
pass = 44444 
(521 schools) 

n = 107894 
pass = 32418 
(381 schools) 

n =7712 
pass = 2662 
(308 schools) 

n =39578 
pass = 17344 
(275 schools) 

n =68828 
pass = 18339 
(251 schools) 

Asian 

2003/04 

n =399 
pass = 291 
(320 schools) 

n =5380 
pass = 3905 
(227 schools) 

n =3903 
pass = 2175 
(213 schools) 

n =248 
pass =119 
(135 schools) 

n =627 
pass = 286 
(96 schools) 

n =2326 
pass = 1595 
(130 schools) 

2004/05 

n = 1202 
pass = 831 
(4 1 7 schools) 

n =7516 
pass = 5350 
(298 schools) 

n =5431 
pass = 2503 
(257 schools) 

n =259 
pass =162 
(125 schools) 

n =762 
pass = 435 
(139 schools) 

n =4399 
pass = 2439 
(165 schools) 

2005/06 

n = 1918 
pass =1414 
(614 schools) 

n = 10863 
pass = 8223 
(407 schools) 

n = 6066 
pass = 3048 
(285 schools) 

n =493 
pass = 357 
(204 schools) 

n =3129 
pass = 2420 
(197 schools) 

n =4492 
pass = 2639 
(174 schools) 

Other 

2003/04 

n =914 
pass = 394 
(320 schools) 

n =2052 
pass = 789 
(227 schools) 

n =849 
pass = 266 
(213 schools) 

n =278 
pass = 33 
(135 schools) 

n =256 
pass = 75 
(96 schools) 

n =221 
pass = 48 
(130 schools) 

2004/05 

n = 1306 
pass = 808 
(586 schools) 

n =2578 
pass =1170 
(358 schools) 

n =2307 
pass = 520 
(312 schools) 

n = 141 
pass = 54 
(204 schools) 

n =319 
pass = 98 
(192 schools) 

n = 1523 
pass = 311 
(210 schools) 

2005/06 

n =2698 
pass =1549 
(762 schools) 

n =3590 
pass =1560 
(521 schools) 

n =2165 
pass = 555 
(381 schools) 

n =522 
pass = 364 
(308 schools) 

n =990 
pass = 526 
(275 schools) 

n = 1543 
pass = 451 
(251 schools) 
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Table 2 (continued) 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Special Education Students 

2003/04 

n =4748 
pass =1451 
(320 schools) 

n =9071 
pass = 1352 
(227 schools) 

n =6874 
pass = 1020 
(213 schools) 

n =993 
pass = 157 
(135 schools) 

n =2797 
pass = 552 
(96 schools) 

n =2526 
pass = 712 
(130 schools) 

2004/05 

n =8864 
pass = 3108 
(431 schools) 

n = 13436 
pass = 2301 
(255 schools) 

n =9772 
pass = 1490 
(242 schools) 

n = 1419 
pass = 41 1 
(142 schools) 

n =3361 
pass = 853 
(118 schools) 

n =5945 
pass = 101 1 
(163 schools) 

2005/06 

n = 16013 
pass = 6538 
(635 schools) 

n =21657 
pass = 4161 
(395 schools) 

n = 10042 
pass = 1679 
(247 schools) 

n =3072 
pass = 1554 
(221 schools) 

n =6847 
pass = 1599 
(208 schools) 

n =6206 
pass = 1056 
(163 schools) 


Limited English Proficiency Students 

2003/04 

n =21867 
pass = 7334 
(320 schools) 

n =33610 
pass = 5226 
(227 schools) 

n =26748 
pass = 4323 
(213 schools) 

n = 1770 
pass = 134 
(135 schools) 

n = 1135 
pass =121 
(96 schools) 

n = 8269 
pass = 829 
(130 schools) 

2004/05 

n =30413 
pass = 14462 
(420 schools) 

n =64655 
pass = 12509 
(239 schools) 

n =63460 
pass = 8991 
(232 schools) 

n =3713 
pass = 438 
(133 schools) 

n = 1363 
pass = 188 
(97 schools) 

n =45470 
pass = 4692 
(150 schools) 

2005/06 

n =31687 
pass = 14782 
(625 schools) 

n =53339 
pass = 10378 
(387 schools) 

n =41657 
pass = 5858 
(249 schools) 

n =3480 
pass = 583 
(2 1 7 schools) 

n = 10503 
pass = 1344 
(196 schools) 

n =23481 
pass = 1757 
(155 schools) 


Note, n = number of students assessed; pass = number of students who “pass” (at or 
above proficient) the state assessment. 
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Table 3 

MSP-MIS Longitudinal Data for Number of Students Assessed and Number of 
Students at or Above Proficient at State Assessments in Mathematics and Science - 
Same Schools Across Years 2003/04, 2004/05, and 2005/06 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

All students 

2003/04 

n = 44409 
pass = 20405 
(245 schools) 

n = 90046 
pass = 32714 
(196 schools) 

n =94878 
pass = 38417 
(192 schools) 

n =9417 
pass = 2747 
(114 schools) 

n = 11099 
pass = 5273 
(57 schools) 

n =44492 
pass = 22814 
(116 schools) 

2004/05 

n =46523 
pass = 26732 
(245 schools) 

n = 110187 
pass = 41361 
(196 schools) 

n = 122847 
pass = 41975 
(192 schools) 

n =9336 
pass = 3290 
(114 schools) 

n = 10873 
pass = 5329 
(57 schools) 

n =75218 
pass = 26111 
(116 schools) 

2005/06 

n =57577 
pass = 33942 
(245 schools) 

n = 119893 
pass = 48170 
(196 schools) 

n = 124088 
pass = 42758 
(192 schools) 

n =9065 
pass = 3480 
(114 schools) 

n = 10750 
pass = 5480 
(57 schools) 

n =77832 
pass = 27048 
(116 schools) 


Males 

2003/04 

n =22576 
pass = 10209 
(241 schools) 

n =45697 
pass = 16398 
(198 schools) 

n =47681 
pass = 19378 
(194 schools) 

n =4741 
pass = 1363 
(114 schools) 

n = 5596 
pass = 2691 
(57 schools) 

n =22278 
pass = 11671 
(116 schools) 

2004/05 

n =23412 
pass = 13263 
(245 schools) 

n =54941 
pass = 20618 
(196 schools) 

n =61216 
pass = 20922 
(192 schools) 

n =4667 
pass = 1635 
(114 schools) 

n =4932 
pass = 2509 
(57 schools) 

n =37567 
pass =13311 
(116 schools) 

2005/06 

n =29084 
pass = 16364 
(245 schools) 

n =59993 
pass = 22244 
(196 schools) 

n =61467 
pass = 21186 
(192 schools) 

n =4484 
pass = 1678 
(114 schools) 

n =4864 
pass = 2575 
(57 schools) 

n =38776 
pass = 13838 
(116 schools) 

Females 

2003/04 

n =21823 
pass = 10190 
(241 schools) 

n =44326 
pass = 16300 
(198 schools) 

n =46886 
pass = 18850 
(194 schools) 

n = 4669 
pass =1384 
(114 schools) 

n =5495 
pass = 2580 
(57 schools) 

n =22119 
pass = 11095 
(116 schools) 

2004/05 

n =22485 
pass = 13100 
(245 schools) 

n =53762 
pass = 20190 
(196 schools) 

n =59444 
pass = 20235 
(192 schools) 

n =4558 
pass = 1584 
(114 schools) 

n =4772 
pass = 2410 
(57 schools) 

n =36432 
pass = 12345 
(116 schools) 

2005/06 

n =27952 
pass = 16092 
(245 schools) 

n =58346 
pass = 22195 
(196 schools) 

n =59793 
pass = 20464 
(192 schools) 

n =4430 
pass = 1705 
(114 schools) 

n =4762 
pass = 2476 
(57 schools) 

n =37791 
pass = 12729 
(116 schools) 
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Table 3 (continued) 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

White 

2003/04 

n = 10257 
pass - 7847 
(241 schools) 

n =24747 
pass = 1 6068 
(198 schools) 

n =20268 
pass = 12526 
(194 schools) 

n =4354 
pass =1921 
(114 schools) 

n =5697 
pass = 3576 
(57 schools) 

n = 12496 
pass = 8974 
(116 schools) 

2004/05 

n =11437 
pass = 8950 
(245 schools) 

n =27964 
pass = 19280 
(196 schools) 

n =23855 
pass = 14450 
(192 schools) 

n =4249 
pass = 2253 
(114 schools) 

n =5707 
pass = 3629 
(57 schools) 

n = 16275 
pass = 10818 
(116 schools) 

2005/06 

n =20064 
pass = 13759 
(245 schools) 

n =35201 
pass = 21672 
(196 schools) 

n =22930 
pass = 14368 
(192 schools) 

n =4121 
pass = 2240 
(114 schools) 

n =5457 
pass = 3641 
(57 schools) 

n = 15995 
pass = 10755 
(116 schools) 

African American 

2003/04 

n =4962 
pass = 1732 
(241 schools) 

n = 10517 
pass = 2583 
(198 schools) 

n =8178 
pass = 2234 
(194 schools) 

n =1122 
pass = 145 
(114 schools) 

n = 2109 
pass = 348 
(57 schools) 

n =5215 
pass = 2395 
(116 schools) 

2004/05 

n =5122 
pass = 2233 
(245 schools) 

n = 15262 
pass = 3617 
(196 schools) 

n = 10659 
pass = 2356 
(192 schools) 

n = 1011 
pass = 202 
(114 schools) 

n = 1921 
pass = 469 
(57 schools) 

n =7516 
pass = 2400 
(116 schools) 

2005/06 

n =4825 
pass = 2015 
(245 schools) 

n = 15947 
pass = 3141 
(196 schools) 

n = 10273 
pass = 2817 
(192 schools) 

n =886 
pass = 255 
(114 schools) 

n =2090 
pass = 487 
(57 schools) 

n =7314 
pass = 2736 
(116 schools) 

Hispanic/Latino 

2002/03 

n = 126 
pass = 90 
(24 schools) 

n =611 
pass = 261 
(15 schools) 

n = 1011 
pass = 419 
(8 schools) 

n =26 
pass = 21 
(9 schools) 

n = 16 
pass = 1 5 
(5 schools) 

n = 19 
pass = 17 
(2 schools) 

2003/04 

n =27653 
pass = 1 0027 
(241 schools) 

n =45166 
pass = 8537 
(198 schools) 

n =59563 
pass = 20152 
(194 schools) 

n =3138 
pass = 402 
(114 schools) 

n =1152 
pass = 434 
(57 schools) 

n =22703 
pass = 9085 
(116 schools) 

2004/05 

n =28005 
pass = 14540 
(245 schools) 

n =56823 
pass = 12247 
(196 schools) 

n =79291 
pass = 21548 
(192 schools) 

n =3180 
pass = 516 
(114 schools) 

n = 1252 
pass = 457 
(57 schools) 

n =44911 
pass = 9872 
(116 schools) 

2005/06 

n =28894 
pass = 15123 
(245 schools) 

n =57 472 
pass =13514 
(196 schools) 

n =81397 
pass = 2 1 666 
(192 schools) 

n =3088 
pass = 620 
(114 schools) 

n = 1375 
pass = 566 
(57 schools) 

n =47921 
pass = 10351 
(116 schools) 

Asian 

2003/04 

n =398 
pass = 290 
(241 schools) 

n =5369 
pass = 3900 
(198 schools) 

n =3888 
pass = 2161 
(194 schools) 

n =248 
pass =119 
(114 schools) 

n =443 
pass = 23 1 
(57 schools) 

n =2290 
pass =1564 
(116 schools) 

2004/05 

n =237 
pass = 180 
(245 schools) 

n =5675 
pass = 4408 
(196 schools) 

n =4281 
pass = 202 1 
(192 schools) 

n =97 
pass = 65 
(114 schools) 

n =341 
pass = 226 
(57 schools) 

n =3574 
pass = 2109 
(116 schools) 

2005/06 

n =459 
pass = 347 
(245 schools) 

n =6305 
pass = 4862 
(196 schools) 

n =4352 
pass = 2150 
(192 schools) 

n =89 
pass = 68 
(114 schools) 

n =326 
pass = 224 
(57 schools) 

n =3525 
pass = 2178 
(116 schools) 

Other 

2003/04 

n =844 
pass = 370 
(241 schools) 

n =2032 
pass = 788 
(198 schools) 

n =828 
pass = 264 
(194 schools) 

n =273 
pass = 33 
(114 schools) 

n = 182 
pass = 53 
(57 schools) 

n =203 
pass = 45 
(116 schools) 

2004/05 

n =735 
pass = 401 
(245 schools) 

n =2078 
pass = 879 
(196 schools) 

n =2113 
pass = 477 
(192 schools) 

n =89 
pass =17 
(114 schools) 

n = 153 
pass = 36 
(57 schools) 

n = 1341 
pass = 277 
(116 schools) 

2005/06 

n =852 
pass = 362 
(245 schools) 

n =2083 
pass = 710 
(196 schools) 

n = 1742 
pass = 390 
(schools) 

n = 122 
pass = 24 
(114 schools) 

n = 131 
pass = 35 
(57 schools) 

n = 1366 
pass = 378 
(116 schools) 
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Table 3 (continued) 



MATHEMATICS 

SCIENCE 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Elementary 

Schools 

Middle 

Schools 

High 

Schools 

Special Education Students 

2003/04 

n =3742 
pass =1111 
(241 schools) 

n = 8013 
pass = 1257 
(198 schools) 

n =6754 
pass = 991 
(194 schools) 

n =825 
pass = 123 
(114 schools) 

n = 1427 
pass = 341 
(57 schools) 

n =2471 
pass = 685 
(116 schools) 

2004/05 

n =3828 
pass = 1277 
(245 schools) 

n =6954 
pass =1251 
(196 schools) 

n =6447 
pass =1110 
(192 schools) 

n =576 
pass = 121 
(114 schools) 

n = 1304 
pass = 246 
(57 schools) 

n =3892 
pass = 647 
(116 schools) 


Limited English Proficiency Students 

2003/04 

n =20830 
pass = 6968 
(241 schools) 

n =32161 
pass = 4817 
(198 schools) 

n =26160 
pass = 4229 
( 1 94 schools) 

n = 1629 
pass = 97 
(114 schools) 

n =349 
pass = 77 
(57 schools) 

n =8247 
pass = 820 
(116 schools) 

2004/05 

n =23348 
pass= 11839 
(245 schools) 

n =51336 
pass = 10777 
( 1 96 schools) 

n =54002 
pass = 7918 
( 192 schools) 

n =2031 
pass =151 
(114 schools) 

n =347 
pass = 100 
(57 schools) 

n =37586 
pass = 3895 
(116 schools) 

2005/06 

n =23501 
pass = 1 1912 
(245 schools) 

n =35138 
pass = 6677 
( 1 96 schools) 

n =33713 
pass = 5076 
(192 schools) 

n = 1794 
pass = 217 
(114 schools) 

n = 436 
pass = 155 
(57 schools) 

n = 17566 
pass = 1332 
(116 schools) 


Note, n = number of students assessed; pass = number of students who “pass” (at or 
above proficient) the state assessment. 
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Table 4 


Longitudinal School Changes in Mathematics Proficiency 


School Year 

Percent Proficient Students 

Effect Size ( ES) of Change 

MSP FOCUS ON MATH 

MSP FOCUS ON MATH 

YES 1 NO 

YES | NO 

Elementary Schools 

Year 2-Year 3 (2003/04-04/05) 

Increase 
ES = +.28 

Decrease 
ES = -.08 

2003/04 

41.39% 
Students: 37,252 
Schools: 160 

69.65% 

7,157 

81 

2004/05 

55.53% 
Students: 38,033 
Schools: 160 

66.09% 

8,490 

85 

Year 2-Year 4 (2003/04-05/06) 

Increase 
ES = +.35 

Decrease 
ES= -.22 

2005/06 

58.95% 
Students: 39,373 
Schools: 160 

58.96% 

18,204 

85 

Middle Schools 

Year 2-Year 3 (2003/04-04/05) 

Increase 
ES= +.05 

Increase 
ES - +.09 

2003/04 

28.82% 
Students: 70,801 
Schools: 151 

63.95% 

19,245 

47 

2004/05 

31.26% 
Students: 91,366 
Schools: 153 

68.02% 

18,821 

43 

Year 2-Year 4 (2003/04-05/06) 

Increase 
ES = +.14 

Decrease 
ES= -.10 

2005/06 

35.14% 
Students: 94,908 
Schools: 153 

59.32% 

24,985 

43 

High Schools 

Year 2-Year 3 (2003/04-04/05) 

2003/04 

39.53% 
Students: 84,574 
147 

48.37% 

10,304 

47 

Decrease 
ES = -. 14 

No Change 

2004/05 

32.89% 
Students: 112,811 
Schools: 145 

48.58% 

10,036 

47 

Year 2-Year 4 (2003/04-05/06) 

Decrease 
ES = -. 15 

Increase 
ES= +.20 

2005/06 

32.44% 
Students: 114,441 
Schools : 145 

58.44% 

9,647 

47 


119 


Dimitrov 


Table 5 


Longitudinal School Changes in Mathematics Proficiency by Gender 


Gender 

School 

level 

MSP 
Focus on 
Math 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

Males 

Elementary 

YES 

40.73 

54.69 

57.90 

.280 

.345 

NO 

69.12 

65.52 

52.68 

-.077 

-.339 

Middle 

YES 

2S.28 

31.15 

34.74 

.063 

.139 

NO 

63.69 

67.73 

45.79 

.085 

-.362 

High 

YES 

39.77 

33.02 

32.40 

-.140 

-.154 

NO 

47.60 

46.85 

57.89 

-.015 

.206 

Females 

Elementary 

YES 

42.07 

56.30 

59.68 

.286 

.354 

NO 

70.18 

66.69 

53.13 

-.075 

-.353 

Middle 

YES 

29.36 

31.16 

35.59 

.039 

.133 

NO 

64.20 

68.34 

47.24 

.087 

-.343 

High 

YES 

39.13 

32.57 

32.12 

-.137 

-.146 

NO 

49.24 

50.38 

59.03 

.023 

.197 
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Table 6 


Longitudinal School Changes in Mathematics Proficiency by Ethnicity 


Ethnicity 

School 

level 

MSP 
Focus on 
Math 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

White 

Elementary 

YES 

78.96 

81.61 

83.56 

.070 

.120 

NO 

73.94 

74.65 

59.92 

No 

change 

-.300 

Middle 

YES 

60.96 

66.5711 

69.9493 

.120 

.190 

NO 

70.26 

73.33 

49.56 

.070 

-.426 

High 

YES 

63.60 

60.51 

61.73 

-.064 

-.039 

NO 

57.10 

60.78 

65.83 

.075 

.180 

African- 

American 

Elementary 

YES 

27.64 

38.57 

45.48 

.233 

.373 

NO 

75.46 

77.86 

12.10 

.057 

-1.394 

Middle 

YES 

15.16 

17.69 

17.69 

.068 

.152 

NO 

69.99 

70.86 

8.69 

.019 

-1.384 

High 

YES 

25.45 

20.82 

23.35 

-.110 

-.049 

NO 

33.46 

28.18 

47.02 

-.114 

.278 

Hispanic 

Elementary 

YES 

35.86 

52.52 

54.17 

.337 

.370 

NO 

48.73 

39.64 

30.12 

-.183 

-.383 

Middle 

YES 

18.31 

20.93 

23.77 

.066 

.134 

NO 

27.29 

33.79 

19.33 

.141 

-.189 

High 

YES 

33.83 

27.10 

26.20 

-.146 

-.167 

NO 

33.94 

30.67 

46.13 

-.070 

.249 

Asian 

Elementary 

YES 

79.08 

75.59 

80.64 

No 

change 

No change 

NO 

66.83 

79.17 

52.44 

No 

change 

-.2945 

Middle 

YES 

62.20 

66.04 

69.18 

.080 

.147 

NO 

84.44 

87.25 

83.69 

.080 

-.021 

High 

YES 

54.50 

45.55 

46.05 

-.179 

-.169 

NO 

59.73 

57.91 

69.56 

-.037 

.206 

Race not 
reported 

Elementary 

YES 

47.60 

39.70 

63.12 

-.167 

.306 

NO 

38.46 

44.29 

98.33 

.118 

1.545 

Middle 

YES 

37.52 

36.34 

32.35 

-.024 

-.109 

NO 

38.65 

44.36 

36.02 

.120 

-.050 

High 

YES 

50.64 

42.30 

40.06 

-.168 

-.213 

NO 

31.48 

43.37 

42.86 

.246 

.236 

Other 

Elementary 

YES 

41.85 

56.35 

50.66 

.291 

.177 

NO 

47.60 

49.48 

29.23 

.038 

-.380 

Middle 

YES 

27.11 

34.16 

37.54 

.153 

.224 

NO 

57.75 

61.45 

27.65 

.075 

-.619 

High 

YES 

32.36 

22.23 

20.91 

-.228 

-.260 

NO 

29.66 

29.41 

56.16 

No 

change 

.543 
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Table 7 

Longitudinal School Changes in Mathematics Proficiency for Special Education and 


Limited English Proficiency Students 


Special 
education 
and LEP 

School 

level 

MSP 
Focus on 
Math 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

Special 

Education 

Elementary 

YES 

25.50 

29.69 

37.97 

.094 

.269 

NO 

42.29 

45.81 

25.57 

.071 

-.356 

Middle 

YES 

10.39 

12.95 

16.06 

.080 

.168 

NO 

31.21 

31.23 

9.75 

No change 

-.550 

High 

YES 

13.94 

17.48 

17.48 

.097 

.0165 

NO 

17.62 

16.20 

30.71 

No change 

.3083 

Limited 

English 

Proficiency 

Elementary 

YES 

33.34 

51.39 

52.24 

.367 

.384 

NO 

38.66 

27.70 

16.92 

-.234 

-.494 

Middle 

YES 

14.49 

20.82 

19.22 

.167 

.127 

NO 

22.72 

25.83 

15.12 

.072 

-.195 

High 

YES 

15.74 

14.48 

14.56 

-.035 

-.0328 

NO 

28.33 

25.61 

36.70 

No change 

.179 
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Table 8 


Longitudinal School Changes in Science Proficiency 


School Year 

Percent Proficient Students 

Effect Size ( ES) 
of Change 

MSP FOCUS ON 
SCIENCE 

MSP FOCUS ON SCIENCE 

YES | NO 

YES | NO 

Elementary Schools 

Year 2-Year 3 (2003/04-04/05) 

Increase 

ES=+.\6 

No Change 

2003/04 

23.28% 
Students: 7,696 
Schools: 96 

55.49% 

1,721 

18 

2004/05 

30.33% 
Students: 7,678 
Schools: 96 

57.96% 

1,658 

18 

Year 2-Year 4 (2003/04-05/06) 

Increase 
ES = +.22 

Increase 
ES= +.13 

2005/06 

33.39% 
Students: 7,473 
Schools: 96 

58.96% 

1,592 

18 

Middle Schools 

Year 2-Year 3 (2003/04-04/05) 

No Change 

Increase 
ES= +.14 

2003/04 

44.43% 
Students: 9,679 
Schools: 51 

68.52% 

1,420 

6 

2004/05 

45.09% 
Students: 9,430 
Schools: 51 

74.64% 

1,443 

6 

Year 2-Year 4 (2003/04-05/06) 

Increase 
ES= +.08 

No Change 

2005/06 

48.48% 
Students: 9,299 
Schools: 51 

66.99% 

1,451 

6 

High schools 

Year 2-Year 3 (2003/04-04/05) 

2003/04 

49.50% 
Students: 41,638 
Schools: 104 

77.22% 

2,854 

12 

Decrease 

ES=-36 

Increase 
ES = +.11 

2004/05 

31.99% 
Students: 71,083 
Schools: 104 

81.62% 

4,135 

12 

Year 2-Year 4 (2003/04-05/06) 

Decrease 
ES = -.36 

Increase 
ES= +.14 

2005/06 

32.07% 
Students: 73,709 
Schools : 104 

82.78% 

4,123 

12 
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Longitudinal School Changes in Science Proficiency by Gender 


Gender 

School 

level 

MSP 
Focus on 
Science 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

Males 

Elementary 

YES 

22.56 

29.52 

31.78 

.159 

.208 

NO 

56.19 

59.33 

64.35 

.064 

.167 

Middle 

YES 

44.95 

46.88 

50.26 

.039 

.106 

NO 

69.77 

74.86 

67.36 

.114 

-.052 

High 

YES 

50.74 

32.82 

33.12 

-.366 

-.359 

NO 

76.87 

80.38 

82.65 

.086 

.144 

Females 

Elementary 

YES 

24.06 

30.16 

33.75 

.138 

.214 

NO 

54.77 

56.48 

59.51 

.034 

.096 

Middle 

YES 

43.93 

46.12 

49.53 

.044 

.112 

NO 

67.28 

74.42 

74.42 

.158 

No change 

High 

YES 

48.24 

30.93 

30.77 

-.356 

-.360 

NO 

77.57 

82.86 

82.90 

.133 

.134 


124 


Intermediate Trends 


Table 10 


Longitudinal School Changes in Science Proficiency by Ethnicity 


Ethnicity 

School 

level 

MSP 
Focus on 
Science 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

White 

Elementary 

YES 

35.01 

45.35 

47.20 

.210 

.250 

NO 

81.91 

85.84 

86.53 

.107 

.127 

Middle 

YES 

57.95 

57.91 

62.18 

No change 

.0864 

NO 

93.63 

93.63 

90.00 

No change 

-.133 

High 

YES 

69.26 

61.76 

62.55 

-.158 

-.142 

NO 

90.72 

91.35 

91.42 

No change 

No change 

African- 

American 

Elementary 

YES 

6.34 

13.37 

22.14 

.240 

.471 

NO 

42.03 

47.45 

50.98 

.109 

.180 

Middle 

YES 

15.84 

22.00 

23.50 

23.500 

.193 

NO 

19.26 

34.21 

22.49 

.340 

No change 

High 

YES 

40.24 

24.57 

30.46 

-.337 

-.205 

NO 

62.4251 

63.7845 

66.7620 

No change 

.091 

Hispanic 

Elementary 

YES 

11.41 

15.08 

17.89 

.108 

.184 

NO 

19.00 

21.29 

30.04 

No change 

.258 

Middle 

YES 

37.31 

35.88 

41.46 

No change 

.085 

NO 

43.06 

46.05 

36.96 

No change 

No change 

High 

YES 

40.00 

21.94 

21.56 

-.394 

-.404 

NO 

64.70 

79.41 

73.53 

No change 

No change 

Asian 

Elementary 

YES 

35.46 

47.50 

56.00 

.250 

.415 

NO 

76.32 

80.70 

84.38 

No change 

No change 

Middle 

YES 

49.74 

62.50 

67.16 

.258 

.355 

NO 

69.09 

81.16 

76.36 

No change 

No change 

High 

YES 

68.30 

58.42 

61.21 

-.206 

-.149 

NO 

66.6667 

82.7586 

85.5422 

No change 

No change 

Race not 
reported 

Elementary 

YES 

45.20 

33.38 

35.80 

-.243 

-.192 

NO 

NO data available 

Middle 

YES 

37.59 | 34.16 | 38.44 

- 072 I N° change 

NO 

NO data available 

High 

YES 

47.38 

39.67 

37.90 

-.156 

-.192 

NO 

NO data 

33.3333 

75.0000 

— 

- 

Other 

Elementary 

YES 

9.96 

14.47 

14.15 

No change 

No change 

NO 

58.33 

46.15 

56.25 

No change 

No change 

Middle 

YES 

28.81 

20.98 

25.40 

No change 

No change 

NO 

40.00 

60.00 

60.00 

No change 

No change 

High 

YES 

21.10 

20.14 

20.14 

No change 

.148 

NO 

75.00 

90.00 

100.0 

No change 

No change 
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Table 11 

Longitudinal School Changes in Science Proficiency for Special Education and 
Limited English Proficiency Students 


Special 
education 
and LEP 

School 

level 

MSP 
Focus on 
Science 

Percent at or above proficient 

Change Effect Size 

Year 2 
2003/04 

Year 3 
2004/05 

Year 4 
2005/06 

Year 2-3 

Year 2-4 

Special 

Education 

Elementary 

YES 

10.44 

13.20 

17.67 

No change 

.210 

NO 

.1485 

37.91 

44.14 

No change 

.281 

Middle 

YES 

32.20 

23.08 

60.00 

No change 

.565 

NO 

38.56 

41.57 

33.86 

No change 

No change 

High 

YES 

27.71 

15.65 

15.48 

-.295 

-.300 

NO 

27.84 

33.98 

33.98 

No change 

.3575 

Limited 

English 

Proficiency 

Elementary 

YES 

4.69 

6.19 

10.23 

.066 

.214 

NO 

11.85 

14.52 

23.08 

No change 

.299 

Middle 

YES 

20.00 

29.54 

33.42 

.222 

.305 

NO 

32.20 

23.08 

60.00 

No change 

.565 

High 

YES 

9.95 

10.35 

7.57 

No change 

-.084 

NO 

NO data 

60.00 

33.33 

NO data 

NO data 
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Table 12 


Growth Trajectories of Schools in Math and Science Proficiency Across Three Years 
2003/04-2005/06) - Relationships Between Initial Status of School Proficiency, Rate 
of Change, and MSP Focus on Math (or Science) 


Subject/School 

level 

Tests ot Model Fit 

Parameter Estimates 

CFI 

TFI 

SRMR 

Initial Status 
on MSP 
Focus 

Rate of Change 
on MSP Focus 

Initial 

Status 

correlated 

with 

Rate of 

Chanee 

MATH 

Elementary 

schools 

.844 

.833 

.079 

0.33* 

-0.04 

-0.30 

MATH 

Middle 

schools 

.959 

.876 

.035 

-1.46** 

0.25* 

-.53* 

MATH 

High schools 

.976 

.927 

.032 

0.37* 

-0.01 

0.09 

SCIENCE 

Elementary 

schools 

.977 

.932 

.026 

-1.63** 

0.01 

0.01 

SCIENCE 

Middle 

schools 

.963 

.888 

.105 

-1.27 

0.35* 

-0.17 

SCIENCE 
High schools 

0.999 

0.999 

.041 

-0.56 

-0.35 

0.25* 


Note. * p < .05; ** p < .01. 
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Table 13 

Correlations Between Teacher Participation in MSP Activities Across Three Years 
(2003/04, 2004/05, 2005/06) and Student Proficiency at the End Year (2005/06) 


Subject/ 

School level 

r 

N 

n 

Mathematics 

Elementary 

.093* 

498 

109,981 

Middle 

.149* 

293 

230,525 

High 

.241** 

286 

162,342 

Science 

Elementary 

.105 

210 

18,292 

Middle 

.027 

209 

67,629 

High 

.473** 

188 

101,692 


Note. N = number of schools (used for the calculation of the correlation coefficient, 
r); n = number of students who have taken the state assessment in these schools; *p < 
.05, ** p < .01. 


128 


Intermediate Trends 



Figure 1. Longitudinal growth model of changes in school math and science 
proficiency across three years (2003/04-2005/06). 
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Figure 2. Bar-graphs for achievement trends (percent of students at or above 
proficient) for schools that have reported data for any of the three years: 2003/04, 
2004/05, and 2005/06. 
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Figure 3. Bar-graphs for achievement trends (percent of students at or above proficient) 
for schools that have reported data for each of the three years: 2003/04, 2004/05, and 


2005/06. 
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Figure 4. MSPs’ focus on mathematics (“No” or “Yes”): Achievement trends for 
schools reporting data all three years (2003/04, 2004/05, and 2005/06). 


Notes : 1. “Focus on Mathematics” means that an MSP’s activities addressed mathematics at that 
grade-span in any of the three years, whether also focusing on mathematics at that grade span or 
not (“Yes” = did focus; “No” = did not focus). 

2. Using a 90% confidence interval (Cl), the changes in percent of students at or above 
proficient in mathematics from 2003/04 to 2004/05 (2005/06) were statistically significant 
except for the change from 2003/04 to 2004/05 for high schools without focus on mathematics. 
The 90% Cl provides a smaller margin of error than a 95% Cl and, despite a slight decrease in 
the level of confidence, increases the chances of detecting changes when they exist. 
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Figure 5. MSPs’ focus on science (“No” or “Yes”): Achievement trends for schools 
reporting data all three years (2003/04, 2004/05, and 2005/06). 


Notes'. 1. “Focus on Science” means that an MSP’s activities addressed science at that grade- 
span in any of the three years, whether also focusing on science at that grade span or not (“Yes” 
= did focus; “No” = did not focus). 

2. Using a 90% confidence interval (Cl), the changes in percent of students at or above 
proficient in science from 2003/04 to 2004/05 (2005/06) were statistically significant except 
for (a) from 2003/05 to 2004/05 for elementary schools without focus on science and (b) from 
2003/04 to 2004/05 (2005/06) for middle schools regardless of their focus (Yes/No) on science. 
The 90% Cl provides a smaller margin of error than a 95% Cl and, despite a slight decrease in 
the level of confidence, increases the chances of detecting changes when they exist. 
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MATHEMATICS: ELEMENTARY SCHOOLS 
Percent of Schools by Direction of Change in Proficiency 
Year 2-Yea r 4 (2003/04-2005/06) 
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Figure 6. Percent of elementary schools by direction of statistically significant change 
in proficiency (at or above proficient) in mathematics from 2003/04 to 2005/06. 


MATHEMATICS: MIDDLE SCHOOLS 
Percent of Schools by Direction of Change in Proficiency 
Year 2-Year 4 (2003/04-2005/06) 
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Figure 7. Percent of middle schools by direction of statistically significant change 
in proficiency (at or above proficient) in mathematics from 2003/04 to 2005/06. 
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MATHEMATICS: HIGH SCHOOLS 
Percent of Schools by Direction of Change in Proficiency 
Year 2-Year 4 (2003104-2005/06) 
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Figure 8. Percent of high schools by direction of statistically significant change in 
proficiency (at or above proficient) in mathematics from 2003/04 to 2005/06. 


SCIENCE: ELEMENTARY SCHOOLS 
Percent of Schools by Direction of Change in Proficiency 
Year 2-Year 4 (2003/04-2005/06) 
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Figure 9. Percent of elementary schools by direction of statistically significant change 
in proficiency (at or above proficient) in science from 2003/04 to 2005/06. 
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SCIENCE: MIDDLE SCHOOLS 
Percent of Schools by Direction of Change in Proficiency 
Year 2-Year 4 (2003/04-2005/06) 
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Figure 10. Percent of middle schools by direction of statistically significant change 
in proficiency (at or above proficient) in science from 2003/04 to 2005/06. 
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Figure 11. Percent of high schools by direction of statistically significant change 
in proficiency (at or above proficient) in science from 2003/04 to 2005/06. 
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Appendix 

WORDING OF MSP-MIS QUESTIONNAIRE ITEMS* REFERENCED IN THE 

PRESENT REPORT 

Student Achievement : 

Item 7g (2002-04) (Item lie (2004-05)): Provide the following 

information about the number of students who took this assessment 
at [NAME OF SCHOOL] during the [INSERT SCHOOL YEAR] 
school year: 

• Number of students at this grade level taking 

assessment during the [INSERT SCHOOL YEAR] 
school year 

• Number of students taking assessment and scoring at 

or above proficient level 


School Participation in MSP Activities (categorical response ): 

Item A (2002-05): Which of the following conditions apply to this school? 
(check all that apply) 

• 30 percent or more of targeted teachers participated in 30 

or more hours of MSP-sponsored activities during the 
[INSERT SCHOOL YEAR] school year 

• 30 percent or more of targeted students were engaged in 

a challenging mathematics or science curriculum that 
was initiated or revised with MSP support during the 
[INSERT SCHOOL YEAR] school year 

• 30 percent or more of targeted students participated in a 

MSP-supported academic enrichment activity during 
the [INSERT SCHOOL YEAR ] school year 

• None of the above conditions apply to this school for the 

[INSERT SCHOOL YEAR] school year 


School Participation in MSP Activities (numeric response ): 

Item 1 (2002-05): Provide the following information about the TOTAL 
number of teachers in [NAME OF SCHOOL] at the beginning of 
the [INSERT SCHOOL YEAR] school year: 

Item 2 (2002-04) (Item 5 (2004-05)): Using the definition for 
“participating teachers” below, provide the following information 
about the number of teachers in [NAME OF SCHOOL] that actively 
participated in your MSP during the [INSERT SCHOOL YEAR] 
school year: 

Definition for “participating teachers”: Those teachers who have 
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participated in 30 or more hours of MSP-sponsored activities 
during a given school year. Examples include teachers who: 1) 
developed or delivered an MSP-sponsored activity to K-12 students 
or other teachers; 2) participated in an MSP-sponsored effort to 
revise math or science curriculum; 3) received MSP-sponsored 
professional development; and/or 4) took part in MSP-related 
learning communities. 

• [Number of] math teachers 

• [Number of] science teachers 


* All items are from the instrument, K-12 District Survey for Comprehensive and 
Targeted MSPs (some item numbers changed from year-to-year). 
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